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DETAILED ACTION 



1 . This action is responsive to communications: amendment filed on 10/1 1/02 to the 
application filed on 11/20/98, priority filed 12/2/97. 

2. Claim 22 is added. 

3. Claims 1-22 are pending in the case. Claims 1, 10, 14, 18 and 22 are 
independent claims. 

Priority 

4. Receipt is acknowledged of papers submitted under 35 U.S.C. 1 19(a)-(d), which 
papers have been placed of record in the file. 



5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

6. This application currently names joint inventors. In considering patentability of 
the claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of 
the various claims was commonly owned at the time any inventions covered therein 
were made absent any evidence to the contrary. Applicant is advised of the obligation 
under 37 CFR 1 .56 to point out the inventor and invention dates of each claim that was 
not commonly owned at the time a later invention was made in order for the examiner to 



Claim Rejections - 35 USC § 103 
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consider the applicability of 35 U.S.C. 103(c) and potential 35 U.S.C. 102(e), (f) or (g) 
prior art under 35 U.S.C. 1 03(a). 

7. Claims 1-21 remain rejected under 35 U.S.C. 103(a) as being unpatentable over 

Sato (JP-8-293039A, 1 1/5/96, filed 4/24/95) in view of Ohba (JP-3-216767A, 9/21/91 , 

filed 1/21/90), the IDSs submitted by Applicants. 

Regarding independent claim 1 , Sato discloses: 

- sequentially providing music control information in correspondence with the 
music to be played, the music control information containing a plurality of types of 
event data associated with the music to be played (abstract: "it is possible to 
generate from music, images having definite shape and motion, such as the 
motion of a human dancing to the music"; [0002]: "the motion of sounds in each 
part of a music is made to correspond to the motion of images having basic 
shape"; [0006]: il musical information is converted into a series of pieces of motion 
information by appropriately switching the motion at every point of sound 
buildup", "motions of the articulated object are applied to the image ....to display 
an animated image"; [0014]: the motions of limbs of the articulated object are 
stored in advance in the motion database and the chords are detected to match 
with the motion data stored in the motion database] [0035]: "it is possible to 
convert the acoustic art, the music, into specific visual art, the motion of 
articulated object having a definite shape, for example the motion of a human 
dancing to the music") 
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- generating a sound in accordance with the music control information to thereby 
play the music ([0006]: a sound buildup point in accordance with the musical 
information and the motions of the animated image) 

- generating a motion image of the object in matching with progression of the 
music ([0014]: the motions of limbs of the articulated object is matched with the 
chord; [0015]: "an articulated object moving in time to the music may be 
displayed using the pre-stored motion data") 

Sato does not disclose explicitly: 

- selecting and setting the types of event data to the movable parts of the object 
such that the respective movable parts correspond to the types of event data 

- controlling the movements of the respective movable parts in correspondence to 
the types of event data contained in the music control information sequentially 
provided 

Ohba discloses: 

- selecting and setting the types of event data to the movable parts of the object 
such that the respective movable parts in correspondence to the types of event 
data (pages 1 and 2, as shown in figures 2 and 3: selecting and setting the 
motions of the legs and the arms, which are movable parts of an object, such that 
the respective movable parts in correspondence to the types of event data, e.g. 
right leg or left leg moving forward or backward, right arm or left arm swinging 
forward or backward) 
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- controlling the movements of respective movable parts in correspondence to the 
types of event data contained in the music control in correspondence to the types 
of event data contained in the music control information sequentially provided 
(page 2: the motions of legs and the motions of arms are generated with 
correspondent parameters; page 3: the shape of the arm of the character is 
transformed so that the character may move the arms; the shapes of the legs 
and the arms are synthesized to form the entire configuration of character for 
outputting the animation; a filter is formed to detect musical scale, stress and 
length of the synthesized sound based on the MIDI signal to generate the 
parameters RQ, RR, and RS correspondent to the change of the arms and legs 
of the animated object according to the music) 
It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Ohba into Sato since Ohba provides the movements of 
respective movable parts of an animated objects according to the MIDI signals and Sato 
provides the music control information for the movable parts of the objects to display the 
movement of the body of a human to the music. The combination of Ohba and Sato 
would enhance the animation display with music by applying the control of the 
movements of an animated object so that said movements are performed along with the 
music. 

Regarding claim 2, which is dependent on claim 1 , Sato discloses preparing a frame of 
the motion image in advance to generation of the sound corresponding to the same 
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data block to generate the prepared frame timely when generating the sound according 
to the same data block used for preparation of the frame (section [0014], ...chords 
detected with the chord detecting section are matched in advance with the motion data 
stored in the motion database..., motion output section visualizes to display the motion 
data generated...)- 

Sato does not disclose analyzing a block of the performance data to perform said 
preparing. However, it would have been obvious to one of ordinary skill in the art at the 
time of the invention was made to have modified Sato to include analyzing a block of 
the performance data into Sato since the block of performance data is considered 
equivalent to the multimedia segment in Sato where the motion video segments, the 
time segments and the sound segments are generated to match with each other. 

Regarding claim 3, which is dependent on claim 1, Sato discloses generating the key 
frames of the motion image in response to the music control information, and generating 
the sub frames inserted between the successive key frames by interpolation to 

smoothen the motion image ([0027]: generating M frames per one motion motion 

data for display are obtained by interpolating a motion data of m-th frame and a motion 
data of (m+1)th frame... generate motions according to the music) 

Regarding claim 4, which is dependent on claim 1, Sato does not disclose generating 
the motion image of an object representing an instrument player and analyzing the 
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audio control information to determine a rendition movement of the instrument player for 
controlling the motion image as if the instrument player plays the music. 
Instead Sato discloses converting music into images by using the stored motions of an 
articulated object and the chord for generating the motion of a human dancing to the 
music (abstract; sections [0033] and [0035]). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have modified Sato to include that the motion object is an instrument 
player for performing the music since a music player is considered equivalent to a 
motion object with movable parts moving according to the music. 

Regarding claim 5, which is dependent on claim 1, Sato does not disclose determining 
the movement of the movable parts of the object, generating the motion image 
according to the motion parameters, and resetting the motion image to revert the 
movable parts to the default positions in matching with the progression of the music. 
Instead Sato discloses the matching of motions of a human dancing to the music 
(abstract; [0005] and [0035]). 

Ohba discloses generating the motion image according to the motion parameters 
(pages 1 and 2, parameter Q associated with the motions of legs, parameter R 
associated with the motions of arms, parameter S associated with the display position of 
the character... transform the shapes of respective parts of the character expressed with 
the shape data in response to the values of parameters Q, R, S). 
Ohba does not disclose determining the movement of the movable parts of the object. 
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However, it would have been obvious to one of ordinary skill in the art at the time of the 
invention was made to have modified Ohba to include "determining the movement ..." 
for the following reason. "Generating the motion image in Ohba inherently includes 
"determining the movement..." since the movements of movable parts of an object 
should be determined to generate a motion image of the object. 
Also, it would have been obvious to one of ordinary skill in the art at the time of the 
invention was made to have combined Ohba into Sata for facilitating the control of 
motions of an objects in accordance to the played music by using the set parameters for 
corresponding motions. 

Regarding claim 6, which is dependent on claim 1, Sato discloses that the 
synchronization signal, which is provided from the sequencer module and which is 
utilized to regulate a beat of music so that the motion image of the object is controlled in 
synchronization with the beat of the music (sections [0008] - [001 1]). 

Regarding claim 7, which is dependent on claim 1, Sato discloses the synchronization 
of the motion video and the music (abstract, sections [0027]-[0029]). However, Sato 
does not disclose providing the music control information containing event data 
specifying an instrument used to play the music. 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have modified Sato to add specifying an instrument used to play the music 
since it was well known that a music must be played by an instrument or a combination 
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of some instruments, and since the music control information is where to include 
information relating to playing the music. 

Regarding claim 8, which is dependent on claim 1, Sato does not disclose setting 
motion parameters effective to determine the movements of the movable parts of the 
object 

Ohba discloses setting motion parameters effective to determine the movements of the 
movable parts of the object (page 2: parameter Q is stored for designating the motions 
of legs, parameter R is stored for designating the motions of arms, ...) 
Ohba also discloses utilizing the motion parameters to control the motion image of the 
object (refer to claim 1 ). 

Sato and Ohba do not disclose controlling the amplitude of the sound to further control 
the motion image such that the movement of each part determined by the motion 
parameter is scaled in association with the amplitude of the sound. 
However, it would have been obvious to one of ordinary skill in the art at the time of the 
invention was made to have modified Sato and Ohba to include controlling the 
amplitude of the sound to further control the motion image such that the movement of 
each part determined by the motion parameter is scaled in association with the 
amplitude of the sound because of the following reason. The amplitude of the sound 
can be controlled by adjusting the volume of the sound. Since the movement of the part 
of the object is determined by a motion parameter, said movement is synchronized with 
the music, and the amplitude of the sound can be adjusted, these features suggest to 
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synchronize the movement of the motion images with the sound in a way that the 
motion images are adjusted to match with the up and down of the sound. In other 
words, the motion images are scaled in association with the amplitude of the sound. 

Regarding claim 9, which is dependent on claim 1, as mentioned in claim 6, Sato 
discloses that the dancer object, which is a motion object, and the music are 
synchronized (abstract, [0035]). Ohba discloses the motion parameters correspondent 
to the movements of a motion object (pages 1-2). 

Sato does not disclose explicitly that the synchronization signal is provided from the 
sequencer module. However, it would have been obvious to one of ordinary skill in the 
art at the time of the invention was made to have modified Sato to include a sequencer 
module since Sato's disclosure of synchronizing the motions of a dancer object to the 
music suggests such module in Sato for performing the function. 

Claims 10-13 are for an apparatus of claims 1-4, and therefore are rejected under the 
same rationale. 

Regarding independent claim 14, Sato discloses: 

- sequentially providing performance data to perform the music and a timing signal 
to regulate progression of the music (abstract, possible to generate from music 
the motion of a human dancing to the music; section [0002], each part of music is 
made to correspond to the motion of images having basic shape; section [0006], 
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musical information and motions of the articulated object are applied to the image 
....to display an animated image; section [0008]; section [0035], motion of a 
human dancing to the music) 

- generating a sound in response to the timing signal and in accordance with the 
performance data to thereby perform the music (section [0006], a sound buildup 
point in accordance with the musical information and the motions of the animated 
image) 

- generating a motion image of the object to response to the timing signal to 
matching with the progression of the music and utilizing the performance data to 
modify the framework in association with the performed music (sections [0014] 
and [0015], the motions of limbs of the articulated object are matched with the 
chord; section [0005], generating timing to switch the motion of the articulated 
object from musical signal, detecting chord from musical signals, and selecting 
motions from motion database based on the chords in time to the timing to switch 
the motion) 

Sato does not disclose providing motion parameters to design a movement of the 
object, and generating a motion image comprises utilizing the motion parameters to 
form a framework of the motion image. 
Ohba discloses: 

- providing motion parameters for the movements of a moving pictures (pages 1 
and 2, as shown in figures 2 and 3.... parameter Q associated with the motions of 
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legs, parameter R associated with the motions of arms, and parameter S 
associated with the display position of the character) 

- utilizing the motion parameters to form a framework of the motion image (page 2, 
transform the shapes of respective parts of the character expressed with the 
shape data in response to the values of the parameters Q, R and S) 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Ohba into Sato to facilitate reproducing a motion object 
where the movement of the object is synchronized with the sound of the music by using 
the motion parameters for each movement of the motion object. 

Regarding claim 15, which is dependent on claim 14, Sato discloses: 

- generating a sound in response to the timing signal and in accordance with the 
performance data to thereby perform the music (section [0006], a sound buildup 
point in accordance with the musical information and the motions of the animated 
image) 

- generating a motion image of the object to response to the timing signal to 
matching with the progression of the music (section [0005], generating timing to 
switch the motion of the articulated object from musical signal, detecting chord 
from musical signals, and selecting motions from motion database based on the 
chords in time to the timing to switch the motion) 

- preparing a frame of the motion image in advance to generation of the sound 
corresponding to the motion image so that the prepared frame can be generated 
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timely when the sound is generated according to the same block used for 
preparation of the frame (section [0014], motions of limbs of the articulated object 
are stored in advance in the motion database, and the kinds of chords detected 
are matched in advance with the motion data stored in the motion database) 
Sato does not disclose analyzing a block of the performance data to perform said 
preparing. However, it would have been obvious to one of ordinary skill in the art at the 
time of the invention was made to have modified Sato to include analyzing a block of 
the performance data into Sato since the block of performance data is considered 
equivalent to the multimedia segment in Sato where the motion video segments, the 
time segments and the sound segments are generated to match with each other. 

Regarding claim 16, which is dependent on claim 14, Sato discloses generating the key 
frames of the motion image in response to the synchronization signal according to the 
motion parameters and the music control information, and generating the sub frames 
inserted between the successive key frames by interpolation to smoothen the motion 

image (section [0027], generating M frames per one motion motion data for display 

are obtained by interpolating a motion data of m-th frame and a motion data of (m+1)th 
frame... generate motions according to the music). 

Regarding claim 17, which is dependent on claim 14, as mentioned in claim 14, Sato 
does not disclose providing motion parameters to design a movement of the object 
representing a player of an instrument, and wherein the step of generating a motion 
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image comprises utilizing the motion parameters to form the framework of the motion 
image of the player and utilizing the performance data to modify the framework for 
generating the motion image presenting the player playing the instrument to perform the 
music. 

Instead, Sato discloses converting music into images by using the stored motions of an 
articulated object and the chord for generating the motion of a human dancing to the 
music (abstract; [0033], [0035]). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have modified Sato to include that the motion object is an instrument 
player for performing the music since a music player is considered equivalent to a 
motion object with movable parts moving according to the music. 
Claims 18-21 are for a machine readable medium of claims 1-4, and therefore are 
rejected under the same rationale. 

8. Claim 22 is rejected under 35 U.S.C. 103(a) as being unpatentable over Sato 

(JP-8-293039A, 11/5/96, filed 4/24/95) in view of Ohba (JP-3-216767A, 9/21/91, filed 

1/21/90), the IDSs submitted by Applicants. 

Regarding independent claim 22, Sato discloses: 

- providing music control information in correspondence with the music to be 
played such that the music control information is arranged into a plurality of 
channels (abstract: "it is possible to generate from music, images having definite 
shape and motion, such as the motion of a human dancing to the music"; [0002]: 
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"the motion of sounds in each part of a music is made to correspond to the 
motion of images having basic shape"; [0006]: "musical information is converted 
into a series of pieces of motion information by appropriately switching the 
motion at every point of sound buildup", "motions of the articulated object are 
applied to the image ....to display an animated image"; [0014]: the motions of 
limbs of the articulated object are stored in advance in the motion database and 
the chords are detected to match with the motion data stored in the motion 
database] [0035]: "it is possible to convert the acoustic art, the music, into 
specific visual art, the motion of articulated object having a definite shape, for 
example the motion of a human dancing to the music") 

- generating a sound in accordance with the music control information to thereby 
play the music ([0006]: a sound buildup point in accordance with the musical 
information and the motions of the animated image) 

- generating a motion image of the object in matching with progression of the 
music ([0014]: the motions of limbs of the articulated object is matched with the 
chord; [0015]: "an articulated object moving in time to the music may be 
displayed using the pre-stored motion data") 

- controlling movements of the respective movable parts in correspondence to the 
channels of the music control information sequentially provided from the 
sequencer module, thereby generating a motion image of the object in matching 
with progression of the music 

Sato does not disclose: 
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- selecting and setting the channels of the music control information to the 
movable parts of the objects such that the respective movable parts correspond 
to the channels of the music control information 

- controlling movements of the respective movable parts in correspondence to the 
channels of the music control information sequentially provided from the 
sequencer module 

Ohba discloses: 

- selecting and setting the types of event data to the movable parts of the object 
such that the respective movable parts in correspondence to the types of event 
data (pages 1 and 2, as shown in figures 2 and 3: selecting and setting the 
motions of the legs and the arms, which are movable parts of an object, such that 
the respective movable parts in correspondence to the types of event data, e.g. 
right leg or left leg moving forward or backward, right arm or left arm swinging 
forward or backward) 

- controlling the movements of respective movable parts in correspondence to the 
types of event data contained in the music control in correspondence to the types 
of event data contained in the music control information sequentially provided 
(page 2: the motions of legs and the motions of arms are generated with 
correspondent parameters; page 3: the shape of the arm of the character is 
transformed so that the character may move the arms; the shapes of the legs 
and the arms are synthesized to form the entire configuration of character for 
outputting the animation; a filter is formed to detect musical scale, stress and 
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length of the synthesized sound based on the MIDI signal to generate the 
parameters RQ, RR, and RS correspondent to the change of the arms and legs 
of the animated object according to the music) 
It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Ohba into Sato to enhance the animation display by 
applying the control of the movements of an animated object so that said movements 
are performed along with the music. 



Response to Arguments 
9. Applicant's arguments filed 10/1 1/02 have been fully considered but they are not 
persuasive. 

Applicants argue that Sato and Ohba do not disclose or suggest providing music 
control data containing multiple types of event data whereby different types of event 
data is selected and designated to correspond with different movable parts of an 
animated object, whereby the videos of the animated object is generated in response to 
a sequence of event data provided. 
Examiner respectfully disagrees. 

Sato discloses providing music control data containing multiple types of event data 
whereby different types of event data is selected and designated to correspond with 
different movable parts of an animated object ([0002]: the sound of music is made to 
correspond to the motion of images having a basic shape; [0005] and [0006]: the 
motions of an articulated object are stored in the motion database, the musical 
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information is converted into a series of pieces of motion information by appropriately 
switching the motion at every point of sound buildup so that the motions of the 
articulated object are applied to the image of the articulated object to display an 
animated image ). 

Sato further discloses that the videos of animated object is generated in response to a 
sequence of event data provided ([0014]: "motions of limbs of the articulated object 
shown in figure 4 are represented with a series of numerical values of position 
coordinates and angles of joints, and stored in advance in the motion database", "joint 
position coordinates or joint angle value series stored as motion data are made to 
change smoothly from the precedent motion by applying a spline processing or the like", 
" a motion section visualizes to display the motion data generated with the motion 
generating section"; [0015]: "as a result, an articulated object moving in time to the 
music may be displayed using the pre-stored motion data "). 
Ohba discloses generating an animation in a real time fashion by generating 
parameters in response to the MIDI signals to transform the input image and to 
emanate the synthesized sound (Summary of the invention). Since Ohba provides 
synthesizing of sounds (page 3, "a filter 26 is formed of an operation..."), Ohba 
suggests to include the music according to the moving of different parts of an object. 
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Applicants argue that Sato and Ohba do not disclose selecting and setting event 
data to correspond with movable parts of an animated object, or generating video 
images in accordance with such setting. 
Examiner respectfully disagrees. 

As mentioned above, since the event data is selected to correspond with movable parts 
of an animated object are disclosed in Sato ([0002]: the sound of music is made to 
correspond to the motion of images having a basic shape; [0005] and [0006]: the 
motions of an articulated object are stored in the motion database, the musical 
information is converted into a senes of pieces of motion information by 
appropriately switching the motion at every point of sound buildup so that the motions of 
the articulated object are applied to the image of the articulated object to display an 
animated image), Sato suggests selecting and setting event data to correspond with 
movable parts of an animated object. 

Ohba discloses selecting and setting event data to correspond with movable parts of an 
animated object (pages 1 and 2, as shown in figures 2 and 3: selecting and setting the 
motions of the legs and the arms, which are movable parts of an object, such that the 
respective movable parts in correspondence to the types of event data, e.g. right leg or 
left leg moving forward). 

Sato further discloses generating video images in accordance with the setting event 
data to correspond with the movements of an animated object ([0014]: " a motion section 
visualizes to display the motion data generated with the motion generating section"; 
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[0015]: "as a result, an articulated object moving in time to the music may be displayed 
usinoi the ore-stored motion data "). 



Conclusion 

10. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 . 1 36(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

1 1 . The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

Aubin (US Pat No. 4,658,427, 4/14/87). 

Falacara et al. (US Pat No. 6,377,263 B1, 4/23/02, filed 8/4/97, priority filed 7/7/97). 
Buxton, Art or Virtual Cinema, Computer Graphics, February 1997, pages 1-2. 
Mackay et al., Video Mosaic : Laying Out Time in a Physical Space, ACM 1994, pages 
165-172. 
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12. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Cong-Lac Huynh whose telephone number is 703-305- 
0432. The examiner can normally be reached on Mon-Fri (8:30-6:00). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Heather Herndon can be reached on 703-308-5186. The fax phone 
numbers for the organization where this application or proceeding is assigned are 703- 
746-7239 for regular communications and 707-746-7238 for After Final 
communications. 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the receptionist whose telephone number is 703-305- 
9000. 



clh 

12/20/02 
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